zstandard: build wheels for riscv64#161
Conversation
Add a riscv64 wheel build for zstandard, modeled on build-numpy.yml, and register the package in ci_scripts/packages.txt. zstandard ships no riscv64 wheels on PyPI (0.25.0 covers every arch except riscv64) and is a common dependency, so riscv64 users build it from source today. riscv64 specifics: - CIBW_BUILD restricted to <py>-manylinux_riscv64 (musl skipped by construction). - CIBW_BUILD_FRONTEND forced to `build`: upstream pins `build[uv]` in pyproject.toml, which fails on riscv64 because uv is absent from the manylinux_riscv64 image. - Checkout ref is the bare version, since zstandard tags are not v-prefixed. Signed-off-by: Bruno Verachten <[email protected]>
The 0.25.0 tag pins enable = ["cpython-freethreading", "cpython-prerelease"] and targets cibuildwheel 3.x. RISE pins cibuildwheel 4.1.0, which removed the cpython-freethreading group (freethreaded builds are default now) and errors on it at config-parse. Strip that group so 4.1.0 can read the config. Also drop the CIBW_BUILD_FRONTEND override: the manylinux_2_39_riscv64 image now ships uv, so let upstream's build[uv] frontend run and confirm it works on riscv64 rather than forcing the plain build frontend. Signed-off-by: Bruno Verachten <[email protected]>
| python: ["cp312", "cp313", "cp314", "cp314t"] | ||
|
|
||
| steps: | ||
| - name: Checkout python-wheels |
There was a problem hiding this comment.
This checkout isn't used, feel free to remove it. It's a pattern I had placed in my numpy workflow and forgot to remove it after I got it working.
| env: | ||
| CIBW_BUILD: ${{ matrix.python }}-manylinux_riscv64 | ||
| # Upstream pins `build-frontend = "build[uv]"` in pyproject.toml, which | ||
| # fails on riscv64: uv is not present in the manylinux_riscv64 image, so |
There was a problem hiding this comment.
is this leftover from before our check for uv in the manylinux image?
| # version. Manual trigger is the only entry point, so checking the ref is | ||
| # enough to gate uploads. | ||
| if: github.ref == 'refs/heads/main' | ||
| runs-on: ubuntu-24.04-riscv |
There was a problem hiding this comment.
This was also a mistake on my end - we don't need to unnecessarily load up the RISC-V Runners here. Can you change it to ubuntu-latest?
| merge-multiple: true | ||
|
|
||
| - name: Publish to GitLab PyPI registry | ||
| uses: ./python-wheels-repo/actions/publish-to-gitlab |
There was a problem hiding this comment.
As Ludovic suggested, you can instead use:
uses: riseproject-dev/python-wheels/actions/publish-to-gitlab@main
in which case the Checkout python-wheels step above can also be removed.
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| python: ["cp312", "cp313", "cp314", "cp314t"] |
There was a problem hiding this comment.
Out of curiosity, have you tried the 3.11 build before? I've updated the draft dev guide to suggest that our full build matrix should be the following, until 3.15 is released:
["cp311", "cp312", "cp313", "cp314", "cp314t"]
|
One thing I forgot to mention completely in the dev guide (maybe this is best placed in a separate doc section and referenced): We needed an open issue for zstandard. I've created: #163 In this case, since you've already submitted a PR upstream and are now waiting for feedback, the label |
…ish, cp311 - Remove the leftover Checkout python-wheels step in build_wheels (unused). - Publish job runs on ubuntu-latest (no need for a riscv64 runner). - Use riseproject-dev/python-wheels/actions/publish-to-gitlab@main and drop the publish job's checkout. - Add cp311 to the build matrix per the updated dev-guide matrix. Signed-off-by: Bruno Verachten <[email protected]>
|
Thanks @threexc, all applied:
|
|
@gounthar you are the first one opening such issue on the repo, thank you for that! What do you think? Maybe @threexc have a different opinion on that? |
We do generally want to closely mirror upstream. I think there's maybe a debate to have here about how different/similar our workflows actually can be, given that we need to substitute a push action and tweak some other things too. It would be good to update the upstream PR with the native builder and point here. |
3 participants
Adds a riscv64 wheel build for zstandard, modeled on
build-numpy.yml. First package I'm adding since converging my own wheel set into RISE, so tell me if I've missed a house convention.Why zstandard. No riscv64 wheels on PyPI (0.25.0 ships every arch except riscv64), and it's a common dependency, so riscv64 users build it from source today. Upstream's release matrix covers aarch64/x86_64/ppc64le/s390x but not riscv64.
What's here.
.github/workflows/build-zstandard.yml: cibuildwheel onubuntu-24.04-riscv, same shape as numpy (build then publish, publish gated onmain).zstandardadded toci_scripts/packages.txt.Porting notes.
CIBW_BUILD: <py>-manylinux_riscv64(manylinux only, musl skipped by construction).enable = ["cpython-freethreading", "cpython-prerelease"]and targets cibuildwheel 3.x. The pinned cibuildwheel 4.1.0 removed thecpython-freethreadinggroup (freethreaded builds are default in 4.x) and errors on it at config-parse, so a small pre-build step strips it. cp314t still builds. Upstream HEAD already dropped it, so this only applies to this tag.v-prefixed (unlike numpy), so the checkout ref is the bare version.build[uv]. The current manylinux_2_39_riscv64 image ships uv, so I'm lettingbuild[uv]run rather than forcing plainbuild, to confirm it works on riscv64 (this is the open question from the dev-guide review on python-wheels: add a development guide #159).Still watching.
ZSTD_WARNINGS_AS_ERRORS=1is set in upstream's cibuildwheel env, so the strict build could trip on a riscv64-specific warning (my local build used plainpip wheel, which doesn't set it). Test extras (test-groups = ["dev"], xdist) are the other likely hurdle.Kept it minimal on purpose (no ccache, unlike numpy) since the build is quick.
Trigger: zstandard:0.25.0
Refs #163